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(57) ABSTRACT 

A method for displaying closed -captioning with audio pro- 
grams. The audio signal data is located and sent to an audio 
signal processor for conversion into an audio program. The 
closed-captioning data is located and decoded to produce a 
textual display synchronized with the audio program. The 
closed-captioning data is displayed on a remote device, 
either as part of a consumer electronics system, or as a stand 
alone network device. The invention also includes one 
example of an apparatus that receives and decodes the 
components of a complex audio signal to produce both the 
audio program and the closed-captioning on the remote 
device. 

9 Claims, 1 Drawing Sheet 
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TEXT DISPLAY ON REMOTE DEVICE Audio signal data and closed-captioning data are located for 

an audio program. The audio signal data is then converted to 
sound to produce the audio program and the closed- 

BACKGROUND OF THE INVENTION captioning data is simultaneously displayed on a remote 

1 Field of the Invention 5 device. r ^ ne rem o te device may be a remote device associ- 

™ . . t . . ,. , . . _ . ated with a consumer electronics system, or a stand alone 

Tins invention relates to display of textual information on remQte coimected to a network , 

a remote device, more particularly to text information . p , 

accompanying audio programs. Another as P ect of the inveQt ">n * an apparatus for 

- „ . . _ . _ . receiving a complex audio signal having both audio signal 

2. Background of the Invention 10 data and closed . captioning data . ^ a * dio signal dat * is 

Hand-held remote control devices are very common, as sent to an audio signal processor. The closed-captioning data 

they accompany just about every type of consumer elec- is S ent to a closed-caption decoder that decodes the data and 

tronics device sold today. These include televisions, VCRs, converts it into a textual display. The display of the text and 

DVD players, set-top boxes, CD players, tuners, and tape the playing of the audio signal are synchronized, 

decks. Indeed, the proliferation of remote control devices 15 

has resulted in the development of programmable remotes BRIEF DESCRIPTION OF THE DRAWINGS 

that can absorb the functions of most of these remotes and _ 

consolidate them into one place. . For a ^re complete understanding of the present invcn- 

. . , , tion and for further advantages thereof, reference is now 

pother advance in remote controls is the remote control made tQ the followin Detaikd Descrip ti on ^kcn in 

with a separate display for controlling these various devices. *> ion ^ ^ acco ^ Drawin m which . 

One example of such a remote control is shown in U.S. Pat. . , - 

No. 5,282,028. The remote control device described in this J FIG * l shows one example of a consumer electronics 

patent has an alphanumeric display, typically an LCD panel, device and a remote d *PWmg closed-captioned lyrics, 

on the remote control device. This device allows the user to FIG * 2 shows one example of a decoder for decoding 

not only control the various devices with a control commu- 25 closed-caption lyrics from an audio signal, 

nication means; it also allows the user to receive information INSCRIPTION OF THP 

visually from the various devices. ™ 

« . . . . . .... . . f PREFERRED EMBODIMENTS 

One example in the patent is the abihty to receive lnfor- 

mation with regard to music being played across the tele- FIG. 1 shows one example of a system that can utilize 

vision audio system. For example, some cable and satellite 30 aspects of the invention. A consumer electronics system 
providers have "cable- CD" channels that broadcast music of comprised of several components, which may include a 

a certain type across a particular channel of the television. television, a CD player, a tape deck, a VCR, a receiver, a 

The information sent to the remote control includes the DVD player, among other components. For purposes of this 

name of the selection, the artist, the time, etc., information discussion, one component will be referred to as an audio 

very similar to that which would be provided by a disc 35 device. 

jockey in a broadcast environment. However, the informa- This device could be one of any of the above devices, as 
tion provided on the remote control is the information well as other types of devices. Its characteristics must 
provided at the television, it is reproduced on the remote include the ability to receive complex audio data. Complex 
control device. ^ aud i 0 dala has as one component audio signal data, which 
The display of textual information with regard to music can be converted into audible sounds, such as the bit stream 
broadcasts is the topic of two other patents. In U.S. Pat. No. from a CD that is sent through a CD player to produce 
5,161,251, a method and apparatus is disclosed that allows music. At a minimum, the complex audio data contains an 
the textual information related to a broadcast piece of music encoded closed-caption file of the audio signal data, oper- 
to be displayed on a display with the musical selection. 45 able to produce any text that is part of the audio signal data, 
Again the information is that related to the specifics of a such asjyjica^aa ^jextual display. 
piece of music, such as title, artist, year, etc. This informa- The format of the closed-captioning data could be one of 
tion is broadcast as an auxiliary signal on the main carrier many types. For example, the format could be SAMI™ 
signal from the broadcast station. A further adaptation of this (Synchronized Accessible Media Interchange) from 
patent is shown in U.S. Pal. No. 5,561,849, in which the 5Q Microsoft Corporation. Any other type of audio synchro- 
lyrics of the musical selection are broadcast at the same nized text file format may be used as well. It will require 
time. The broadcast could be from a broadcast center, such some sort of synchronization between the musical play 
as an FM transmitter. The lyrics are combined with the audio sequence and the text, such as a time stamp that indicates 
broadcast and sent as auxiliary data on the main carrier when a given portion of text is to be displayed. The time 
signal. A tuner that displays both the tuning information on 55 stamp will more than likely be relative to the time of musical 
one display and the lyrics on another display receives the play. 

lyrics. Methods are provided for coordination between the Referring again to FIG. 1, it can be seen that the consumer 

lyncs and the music. electronics system 10 has connection with a remote device 

However, these approaches do not provide for any means 12. The connection between the system 10 and the device 12 

of receiving the lyrics or textual information from other 60 could be via a cable 14. However, the connection cpuld^be^ 

sources than the broadcast message. Additionally, the dis- Yii£XE£§I^ito^ 

play of the lyrics is limited to the tuning device that receives Jedinology^ For example, the porTliSlCor the remote device 

the signal. could be part of one of the components of the system. The 

ci a d v nc -rue im/cMnnM component having the port is not limited to the audio player 

SUMMARY OF THE INVENTION 65 although that may be the most sensible approach. 

One aspect of this invention is a method of displaying text Further, the remote device 12 does_ju i Lnecessarilv cod-_ 

corresponding to closed-captioning for audio programs. note a remote control device, in that the device may not 
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actually have control functions for operating the consumer 
electronics system 10. The remote devicecou l d fre a periph - 
eral part of _the_c o nsumer_elec tronics jevice fhat is specifi- 
cally provided for such textual displays. However, for famil- 
iarity reasons, the remote device of FIG. 1 is shown as a 
remote control device with a screen for displaying the 
captioned text. 

One possibility for the consumer electronics system could 
be an Internet, or other networked, server located outside the 
home, of which the remote device is the only part of the 
system actually seen by the consumer. The_ serv er could,, 
do^loadjnusj^ oj; oth^r_network ajid x*^y~ 

the remote device. In this case, all parts of the apparatus 
necessary to perform this invention would be contained in 
the remote device 

The discussion has focused on the overall system and its 
operation, whether it is a part of an in-home consumer 
electronics device, or a remote server. Referring now to FIG. 
2, it is possible to see one implementation of an apparatus to 
decode the audio signal data for synchronization with the 
closed -captioning data. 

In this implementation, the complex audio data is 
received as one stream at the audio player 20. However, the 
two components could be received as two separate streams. 
In this example, the audio play has a decoder that splits the 
complex stream into its individual components and send the 
audio signal data to the audio signal converter 28 that 
converts the audio signal into sounds heard by the consumer. 
The closed-captioning data is sent to a caption decoder 24 
that converts the data into textual display. The timing signals 
may be sent only to the caption decoder, only to the audio 
signal converter, or to both. 

Once the synchronization, conversion and decoding are 
complete, the two signals are sent to the appropriate portion 
of the consumer electronics device. In this case, the audio 
signal data converted into sound signals is sent to the 
speakers as an audio program. The closed-captioning text is 
sent to the remote device for display. 

One specific example would be if the lyrics or textual part 
of the complex audio signal were encoded using the SAMI 
format. The complex audio signal would constitute the 
SAMI file and the audio data. The audio device of FIG. 2 
could be part of a CD player. In this instance, the CD player 
would comprise the audio player 20, the caption decoder 24 
and the audio signal processor 28. In this case the timing 
information would be the time stamp within the SAMI file. 
The caption data would need to be decoded and sent to the 
remote device as the audio device tracks the playing time of 
the audio signal. In this case, the timing signal does not need 
to be sent to the audio signal processor, one of the options 
for the timing information mentioned above. 

This invention allows the hearing impaired to see the 
lyrics to accompany the music to which others are listening. 
Alternately, it could allow them to read a pre-recorded 
speech that was encoded with the synchronized closed- 
captioning. In cither instance, or in any of a number of 
examples, this invention allows a person to read the textual 
content of an audio program. 
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It should be noted here that although a television is shown 
as part of a consumer electronics system, the television is not 
needed for implementation of this invention. Topically, 
however, the television is the device that requires a remote 
control device. The remote control device of a television is 
only one example of a remote device to be used for dis- 
playing the closed-captioning discussed above. 

Thus, although there has been described to this point a 
particular embodiment for a method and structure for 
closed-captioning data for an audio program to be displayed 
on a remote device, it is not intended that such specific 
references be considered as limitations upon the scope of 
this invention except in -so -far as set forth in the following 
claims. 

What is claimed is: 

1. A method for displaying closed-captioning data for an 
audio signal on a remote device, comprising the steps of; 

locating and decoding audio signal data; 

locating and decoding closed-captioning data, wherein the 

closed-captioning data and the audio signal data come 

from separate sources; 
playing the audio signal data, thereby producing an audio 

program; 

coordinating timing of the closed-caption data to synchro- 
nize the closed-captioning data with the audio program; 
and 

displaying the closed-captioning data on the remote 
device. 

2. The method as claimed in claim 1 wherein said audio 
signal data is located on a compact disc. 

3. The method as claimed in claim 1 wherein said audio 
signal data is located on a networked server. 

4. The method as claimed in claim 1 wherein said 
closed-captioning data is located on a networked server. 

5. The method as claimed in claim 1 wherein said remote 
device further comprises a remote control for a consumer 
electronics system. 

6. The method as claimed in claim 1 wherein said remote 
device further comprises a remote network device. 

7. An apparatus for displaying closed-captioning data for 
an audio signal on a remote device, comprising: 

an audio device operable receive to an audio signal and a 
closed-captioning component, wherein the closed- 
captioning component and the audio signal come from 
separate sources; 

a closed-captioning decoder, operable to convert the 
closed-captioning component into a textual display; 

an audio signal processor operable to produce an audio 
program from the audio signal, and to synchronize the 
closed-captioning component to the audio signal; and 

a remote device to display the textual display. 

8. The apparatus as claimed in claim 7 wherein said audio 
device, said closed-captioning decoder and said audio signal 
processor are all contained in a single piece of equipment. 

9. The apparatus as claimed in claim 7 wherein said 
remote device contains said audio device, said closed- 
captioning decoder and said audio signal processor. 
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